Adopting multiple vision transformer layers for fine-grained image representation

Abstract Accurate discriminative regions proposal has an important effect for fine-grained image recognition. The vision transformer (ViT) brings about a striking in computer duo to its innate muti-head self-attention mechanism. However, the attention maps are gradually similar after certain layers and since ViT adds classification token perform classification, it is unable effectively select patches classification. To accurately detect regions, we propose novel network AMTrans, which efficiently increases learn diverse features utilizes integrated raw capture more salient feature. Specifically, employ DeepViT as backbone solve collapse issue. Then, fuse each head weight within layer produce map. After that, alternatively use recurrent residual refinement blocks promote feature detection then utilize semantic grouping method region. A lot of experiments prove that AMTrans acquires SOTA performance on three widely used datasets under same settings, involving Stanford-Cars, Stanford-Dogs CUB-200-2011.